
## Usage

### 1. Dataset Preparation for ImageNet Experiment 

#### In-distribution dataset

##### ResNet50:
Run python imagenet_resnet/feature_extraction.py

Run python imagenet_resnet/feature_sort_resnet_50.py

Place the sorted features in the folder of cache/imagenet_feature_resnet_50_sorted

##### MobileNetv2:
Run python imagenet_mobilenet/feature_extraction.py

Run python imagenet_mobilenet/feature_sort_mobilenet_v2.py

Place the sorted features in the folder of cache/imagenet_feature_mobilenet_v2_sorted


#### Out-of-distribution dataset

We have curated 4 OOD datasets from 
[iNaturalist], 
[SUN], 
[Places], 
and [Textures], 
and de-duplicated concepts overlapped with ImageNet-1k.

For iNaturalist, SUN, and Places, we have sampled 10,000 images from the selected concepts for each dataset,

Please put all downloaded OOD datasets into `./datasets/ood_data`.

### 2. Dataset Preparation for CIFAR Experiment 

#### In-distribution dataset

The downloading process will start immediately upon running. 

#### Out-of-distribution dataset


We provide links and instructions to download each dataset:

* [SVHN] download it and place it in the folder of `datasets/ood_data/svhn`. Then run `python select_svhn_data.py` to generate test subset.
* [Textures]: download it and place it in the folder of `datasets/ood_data/dtd`.
* [Places365]: download it and place it in the folder of `datasets/ood_data/places365/test_subset`. We randomly sample 10,000 images from the original test dataset. 
* [LSUN]: download it and place it in the folder of `datasets/ood_data/LSUN`.
* [iSUN]: download it and place it in the folder of `datasets/ood_data/iSUN`.
* [LSUN_fix]: download it and place it in the folder of `datasets/ood_data/LSUN_fix`.



### 3.  Pre-trained model

Pre-trained models are placed in the /checkpoints folder.

## Preliminaries
It is tested Python 3.4 environment, and requries some packages to be installed:
* [PyTorch 1.2.0]
* [scipy]
* [numpy]
* [sklearn]
* [ylib](has been given in the code)

## Demo
### 1. Demo code for Cifar-10 Experiment 
Run python feat_extract.py --in-dataset CIFAR-10  --out-datasets SVHN LSUN LSUN_resize iSUN dtd places365 --name densenet  --model-arch densenet --epochs 1002

Run python run_cifar_densenet.py --in-dataset CIFAR-10  --out-datasets SVHN LSUN LSUN_resize iSUN dtd places365 --name densenet  --model-arch densenet

### 2. Demo code for Cifar-100 Experiment 
Run python feat_extract.py --in-dataset CIFAR-100  --out-datasets SVHN LSUN LSUN_resize iSUN dtd places365 --name densenet  --model-arch densenet --epochs 1002

Run python run_cifar_densenet_c100.py --in-dataset CIFAR-100  --out-datasets SVHN LSUN LSUN_resize iSUN dtd places365 --name densenet  --model-arch densenet

### 3. Demo code for ImageNet Experiment on ResNet50
Run python feat_extract_largescale.py --in-dataset imagenet  --out-datasets inat sun50 places50 dtd  --name resnet50  --model-arch resnet50

Run python run_imagenet.py --in-dataset imagenet  --out-datasets inat sun50 places50 dtd  --name resnet50  --model-arch resnet50

### 4. Demo code for Cifar-100 Experiment on MobileNetv2
Run python feat_extract_largescale.py --in-dataset imagenet  --out-datasets inat sun50 places50 dtd  --name mobilenetv2  --model-arch mobilenetv2

Run python run_imagenet_mobilenet.py --in-dataset imagenet  --out-datasets inat sun50 places50 dtd  --name mobilenetv2  --model-arch mobilenetv2


